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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
BEFORE THE BOARD OF PATENT APPEALS AND iNTERFERENCEd 



In re Application of 

GERARD DE HAAN ET AL. 
Serial No,: 09/855,628 
Filed: May 15, 2001 



Docl^et 



Atty. 

PHNL 0100 
Group Art Unit: 261 
Examiner: B.M. Senf: 



MOTION ESTIMATOR FOR REDUCING HALOS IN MC UP-CONVERSION 

Commissionsr for Patents 
P.O- Box 1450 

Alexandria, VA 22313-1450 



Sir: 



APPEAL BRIEF 



This is an appeal from the Examiner of Group 2613 
rejecting claims 1^25 in this application. 



f inaJ ly 



(i) Real Party in Interest 

The real party in interest in this application is KONI]^|KLIJKE 
PHILIPS ELECTRONICS N.V, by virtue of an assignment from the 
inventors recorded on July 30, 2001, at Reel 012032^ Frames 697- 
698. 



(ii) Related Appeals and Interferences 

There are no other appeals and/or interferences relatec 
this application* 
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(iii) Status of the Claims 
Claims 1-25 stand finally rejected by the Examiner. 

(iv) Status of Amendments 
There was one (1) Response filed on March 14, 2005, aftfer 

final rejection of the claims on February 16, 2005, this Response 
having been considered by the Examiner. 

(v) Summary Of Claimed Subject Matter 
The invention relates to a method and apparatus for defecting 

motion in an image signal at a temporal intermediate p.ositicn 
between previous and next images, in which a criterion function for 
candidate vectors is optimized, this function being dependert on 
data from both previous and next images, and in which the 
optimizing is carried out at the temporal intermediate positiion in 
non-covering and non-uncovering areas* 

Appellants have found that an estimator estimating motijon 
between two successive pictures from a video sequence, cannct 
perform well in areas where covering or uncovering occurs. Ilhis is 
due to the fact that it is typical for these areas that the 
information only occurs in either one or the other of the tvjo 
images- Block matchers, as a consequence, will always find Jjarge 
match errors even for the correct vector. 



NL010094-BRIEF-070105 



PAGE4/24 ^ RCVD AT 7/5/2005 12:37:07 PM [Eastern Dayli^^ 



07/05/2005 11:41 



8456233933 



GOODMAN 



PAGE 05/24 



The subject invention seeks to overcome this problem by 
carrying out the optimization at the temporal position of the next 
image in covering areas and at the temporal position of the 
previous image in uncovering areas - 

As described in the specification on page 11, line 27 page 
12, line 11, and shown in Fig. 6, the apparatus comprises m^ans 1 
for optimizing a criterion function for candidate vectors, image 
signal J being applied to the optimizing means 1. In additi4n^ the 
image signal J is also applied to means 2 for detecting 
covering/uncovering areas in the image signal J. An output from the 
detecting means 2 is applied to the optimizing means 1 for 
switching the functioning of the optimizing means such that [the 
optimizing is carried out, instead of at the temporal inteniediate 
position, at the temporal position of the next image in covering 
areas ^ and at the temporal position of the previous image ix 
uncovering areas. 

As described on page 12, lines 18-34, a retimer 3 deteifmines a 
velocity edge Xg; and marks an occlusion area around the edgd in 

response to a foreground velocity and a background velocity 
determined in the image signal J by a foreground/background 
detector 4. The retimer 3 controls the optimizing means 1 siich that 
in the occlusion area, a foreground velocity is replaced by 
background velocity or vice versa, depending on whether the 
occlusion is a covering or uncovering area, the sign of the 

NL010094-BRIEF-070105 3 
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foreground velocity^ and on which side of the velocity edge the 

foreground is located. 

As shown in Fig. 9, and described in the specification on page 
13, lines 1-24, the foreground/background detector 4 includes 
calculating means 5 and 6 for calculating, from the image s: 
in response to a covering/uncovering detector 1, a. first 
and a second position, and, in response thereto, calculatincf 
8 for calculating a third intermediate position. The image 
and the third intermediate position are used by fetching 
derive the background velocity VgQ. 

Alternatively, as shown in Fig. 10 and described in thet 
specification on page 13, line 24 to page 14, line 8, the 
apparatus, and more particularly, the foreground/background 
detector 4 includes projecting means 10 and 11 coupled to receive 
the image signal J and controlled by a covering/uncovering detector 
12, The projecting means projects two positions on either si|de of 
the velocity edge Vj^. In response to the projected two posii^ions, 

an identification means 14 identifies a background velocity V-qq and 
projects to a foreground velocity Vpg. The f oreground/backgijound 
detector 4 may include verification means, i.e., checking m^ans 13, 
which checks if the two projections yield the same vector 

Alternatively, as shown in Fig* 11 and described in the 
specification on page 14, lines 9-27, the apparatus, and morje 
particularly, the foreground/background detector 4 includes, along 
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with a covering/uncovering detector 11 ^ projecting means 16; 
receiving the image signal J, for projecting^ under control [of the 
covering/uncovering detector 17, the discontinuity to the p^fevious 
vector field in the covering situation and to the future vector 
field in the uncovering situation- In addition^, velocity 
determining means 18 and 19 are included for determining thv. 
velocity on opposite sides of the discontinuity* Testing me4ns 20, 
coupled to outputs of the projecting means 16 and the velocity 
determining means 18/19, derives therefrom the background vector 
Vbg- 

Finally, as shown in Fig* 12 and described in the 
specification on page 14, line 28 to page 15, line 2, an imdge 
display apparatus is shown including apparatus 21 for. detect ing a 
motion vector in accordance with that shown in Fig, 9, 
interpolating means 22 coupled to an output of the apparatud 21, 
and a display device 23 connected to an output of the interpolating 
means . 



(vi) Grounds of Rejection to be Reviewed on Appeal 

(A) The invention, as' claimed in claims 1-6, 13-18 and 25, 

rejected under 35 U*S.C. 102(e), as being anticipated 

Patent 6,594,313 to Hazra et al. 
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(B) The invention, as claimed in claims 7-12 and 19-24^ stands 
rejected under 35 U.S.C- 103(a), as being unpatentable over 
Hazra et al. in view of U.S. Patent 6,480,615 to Sun al. 

(vii) Arguments 

(A) The Hazra et al, patent discloses increased video playl}>ack 
framerate in low bit-rate video applications, in which an 
interpolated frame intermediate of a frame pair is synthesized 
based on a variety o^ methods - 

The subject invention, as claimed in claim 1, relates tjo a 
method for detecting of motion at a temporal intermediate position 
between a previous and a next image, in which the method conjprises 
optimizing a criterion function for candidate vectors, said 
criterion function depending on data from both the previous and 
next images, the optimizing being carried out ""^at the tempoijal 
intermediate position in non-covering and non-uncovering are[as". In 
particular, the method of the subject invention, as claimed in 
claim 1, comprises that "tho optimizing is carried out at thie 
temporal position of the next image in covering areas and at the 
temporal position of the previous image in uncovering areas 

Appellants would like to point out that it is well-f ounlded 
that ^^A claim is anticipated only if each and every element |as set 
forth in the claim is found, either expressly or inherently 
described, in a single prior art reference." verdegaal Bros 

NL010094-BRIEF-O701O5 6 
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Union Oil Co. of Californiar 814 F,2d 628, 631, 2 USPQ2d 10$1, 1053 
(Fed. Cir. 1987), and ^^The identical invention must be showA m as 
complene detail as is contained in the ... claim- Richardshn v, 
Suzuki Motor Co., 868 F.2d 1226, 1236, 9 USPQ2d 1913, 1920 tFed. 
Cir, 1989) . 

The Examiner has indicated in the rejection that the a$ove 



limitations may be found in Hazra et al. at col, 1, lines 



5::-66. 



This portion of Hazra et al. states: 



""^Additionally, the best-matched block for each 
interpolated block is selected from the current winning 
block for each list based on an error criterion and an 
overlap criterion. The interpolated frame is 
synthesized based on the best-matched block for each 
interpolated block • 

"In another embodiment, a method includes 
selecting a zero motion vector for a given pixel in an 
interpolated frame upon determining a current pixel in 
a current frame corresponding to the given pixel in th^ 
interpolated frame is classified as covered or 
uncovered- The interpolated frame is synthesized based 
on selecting the zero motion vector for the given pixel, 
in the interpolated frame upon determining the current 
pixel in the current frame corresponding to the given 
pixel in the interpolated frame is classified as 
covered or uncovered."' 



Appellants submit that while Hazra et al, mentions the terms 
^^covered" and '^uncovered", a careful reading of the above will 
indicate that there is not disclosure in Hazra et al. ""'that the 
optimizing is carried out at the temporal position of the n^xt 
image in covering areas and at the temporal position of the 
previous iro^age in uncovering areas." Rather, Hazra et al. 
specifically states ^^the interpolated frame is synthesized leased on 
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selecting the 3:;ero motion vector for the given pixel in the 

interpolated frame upon determining the current pixel in th4 

current frame corresponding to the given pixel in the interpolated 

frame is classified as covered or uncovered," 

In response thereto the Examiner now states: 

"Hazra ^313 (figs. 3 and 4) discloses, process for 
determining best motion vector {candidate vectors, besi: 
match) between the current/new image/frame (as coverincf 
area) , which has its own vector (which also consider 
next image with respect to the previous image) and 
previous image/frame (as uncovering area), which are 
situated temporally between the f rames /blocks . The 
best-matched vector is being selected, is in fact a 
process of optimizing*'' 

Appellants submit that the Examiner is mistaken. In 
particular. Fig. 3 is a block diagram showing merely 4 bloc) 
wit;, classifying pixels 300, dividing the framing into blocl: 
determining a best motion vector 304 and synthesizing the 
interpolated frame 306. Fig. 4 merely shows a previous frame: 
particular block 4 06, a current frame in which, due to motl 
corresponding block is 410, and the interpolated frame where 
corresponding block is 408. It should be apparent that these 
figures do not show that in non-uncovered and non-covered 
optimizing is carried out at the temporal intermediate posit 
covering areas, optimizing is carried out at the temporal pc 
of the next image, and in uncovering areas, optimizing is carried 
out at the temporal position of the previous image. 



lie n 
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Further^ from the Examiner's statement^ it is apparent 
there is a misunderstanding of the meaning of the terms 
and ^^uncovering''- In particular, the terms ^^covering" and 
"T^ncovering" do not mean the current image and the previous 
respectively. Rather, ^^covering" and ""^uncovering" are relat 
motion vectors in the vicinity of an edge- (Appellants had 
attempted to further illustrate the difference between 
and ^'uncovering" by submitting a copy of WO 00/11863 cited 
3 and 12 of the subject specification. However, this re 
not considered by the Examiner*] 

In Hazra et al*, use is made of the forward motion vect; 
the backward motion vector. However, in the event that a 
classified as either covered or uncovered, a zero motion ve 
chosen" (see col- 6, lines 63-67). As such, Hazra et al. 
does not disclose the limitation "that the optimizing is 
out at the temporal position of the next image in covering 
and at the temporal position of the previous image in uncove 



image, 
d to 



cov<iring' 



on 



5f erence 



pi::el 



def i n 



areas . 



Claim 3 states ^^the criterion function is a match erroi which 
is minimized*" 

The Examiner indicates that this is shown in Hazra et a|l* at 

col- 6, lines 25-35, which states: 

'^In FIG. 5, block 514 is performed in one 
embodiment by the method of FIG. 8, as the final motion 
vectot is selected from one of the candidate lists. In 
FIG. 8 in block 808, the selection criterion from. among 
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tho three candidates. Forward Motion Vector (FMV) 
Candidate 802, Backward Motion Vector (BMV) Candidate 
804 and Zero Motion Vector (ZMV) Candidate 806, from 
the candidate lists uses both the block matching error 
(MAD or the Sum of Absolute Difference (SAD) ) and the 
overlap to choose the best motion vector. The rational^ 
for using the block matching error is to penalize 
unreliable motion vectors even though they may result 
in a large overlap." 



error 



the 



-irig 



Appellants submit that it should be clear from the abo^ 
while Hazra et al - arguably does perform a block matching 
choose the best motion vector, this is not what is done in 
invention as claimed. Rarher, claim 1 states that ^'optimiz 
the criterion function) is carried out at the temporal posit, 
the next image in covering areas and at the temporal position 
the previous image in uncovering areas , " 

Appellants submit that it appears that Hazra et al. is 
performing a block match error analysis on all the motion 
candidates as opposed to, selectively, the forward motion 
the backward motion vectors, and the zexo motion vectors. Ir 
subject invention, depending on whether there is covering ox 
uncovering, or areas of non-uncovering or non-covering, 
optimization is performed at differing temporal positions, i 
the temporal position of the (intermediate) interpolated ima 
temporal position of the next image, or the temporal position 
the previous image. 
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(B) The above arguments with regard to Hazra et al* are 
incorporated herein by reference. 

The Sun et al. patent discloses motion estiination with; 
sequence of data frames using optical flow with adaptive gr^dients^ 
in which Sun et al. ""'xnentions" the terms ^'occlusions" and 
(at col- 3^ lines 5^50) « 

However, Appellants submit that Sun et al. neither sho^s nor 
suggests determining a velocity edge, marking an occlusion 
around the determined edge, and replacing the foreground ve] 
by background velocity or reversibly dependent on whether ttj' 
occlusion is a covering or uncovering area, the sign of the 
foreground velocity and on which side of the ve.l ocity edge 
foreground is (as specifically claimed in- claim 7) • 

Furthermore, since the rejection is the combination of 
et al» and Sun et al,. Appellants submit that Sun et al. 
supply that which is missing from Ha^ra et al., i.e., ^'that 
optimizing is carried out at the temporal position of the n^t 
image in covering areas and at the temporal position of the 
previous image in uncovering areas . " 



the 
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Based on the above arguments, Appellants believe that tie 
subject invention is neither anticipated nor rendered obvious by 
the prior art and is patentable thereover. Therefore, Appellants 
respectfully request that this Board reverse the decisions of the 
Examiner and allow this application to pass on to issue . 



Respectfully submitted. 




Attorney 
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(viii) Claim Appendix 

CLAIMS ON APPEAL 
1. (Amended) A method for detecting motion at a temporal 

intermediate position between previous and next images, in wL 
criterion function for candidate vectors is optimized, said 
function depending on data from both previous and next imagesi 
in which the optimizing is carried out at the temporal 
position in non-covering and non-uncovering areas, character:, 
that the optimizing is carried out at the temporal position 
next image in covering areas and at the temporal position of 
previous image in uncovering areas. 



Lich a 



interr\ediat 



of 



2, (Amended) The method as claimed in claim X, wherein the 

previous image is shifted over a fraction a times the 
vector, the next image is shifted over 1 -a times the 
vector and the fraction a may change within the image period 



candidate 



candidate 



3, (Amended) The method as claimed in claim 1, wherein 
criterion function is a match error which is minimized. 

4. (Amended) The method as claimed in claim 2, wherein 
fraction a is controlled by a covering/uncovering detector 
matching process. 



and 

e 

zed in 

the 
the 



:he 



the 
the 
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5. (Amended) The method as claimed in claim 4, wherein the 
fraction a is set to l in case of covering and set to 0 in cajse of 
uncovering , 

6. (Amended) The method as claimed in claim 4, wherein t^he 
covering/uncovering detector decides on data in a previous i^age to 
the fraction a in the current estimation. 

7. (Amended) The method as claimed in claim 1, wherein cl 
velocity edge Xe is determined, an occlusion area is marked around 
said edge, and in said occlusion area, foreground velocity 
replaced by background velocity or reversibly dependent on 

5 the occlusion area is a covering or uncovering area, the sign of 
the foreground velocity and on which side of the velocity ed^e Xe 
the foreground is* 

8. (Amended) The method as claimed in claim 7, wherein the 
position of a velocity edge 

• a first position in the previous (covering) or next 
(uncovering) image is c'alculated by shifting over the fir^t 

5 vector at one side of the edge 

• a second position in the previous (covering) or next 
(uncovering) image is calculated by shifting over the sec<})nd 
vector at the other side of the edge 
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• and a third intermediate position between and is 
10 calculated 

• while finally, the vector fetched with v-^ at the third po£ 
in the previous (covering) or next (uncovering) image is 
those regions of the image in the environment of the edge/ to 
no vector is projected, in case the background vector v^^ 



filled 



it ion 

in 
which 

shofuld be 



15 filled in, and the vector chosen between 



*-(:) 



,n) and Z3(x + 



V 



which is most different from v-^ is filled in, in caae a foreground 



vector should be filled in. 



9 . (Amended) 



The method as claimed in claim 8, wherein ;he 



intermediate position is (5^ + x^)/2. 

10. (Amended) The method as claimed in claim 1, wherein 
background velocity is identified as. a velocity which crosses 
velocity discontinuity and projects to a foreground velocity 
previous picture, whereas a foreground velocity projects to 



11. (Amended) The method as claimed in claim 7, wherein 
edges it is tested whether the mentioned edge has moved over 
first vector on one side of the edge, or over the second 
the other side of the edge, in case the edge moves with the 



the 
in the 
itself. 



near 
the 

on 
first 



vector 
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(second) vector, the second (first) vector is Eilled in those 
regions of the projected vector field in the environment of the 
edge, to which no vector is projected, in case a background \ 



v^^ should be filled in, and the other vector is filled in, ijn 
a foreground vector v^^ should be filled. 



12. (Amended) The method as claimed in claim 10^ wherein the 
crossing from a background region to a foreground region in l^he 
previous image is verified by the match error o£ the vector in that 
block- 

13, (Amended) An apparatus for detecting motion at a tem|>oral 
intermediate position between previous and next images, comp:f±slng 
means for optimizing a criterion function for candidate 
said function depending on data from both previous and next ^mages 
in which the optimizing is carried out at the temporal i 
position in non-covering and non-uncovering areas, character 
that said apparatus further comprises means for detecting co' 
or uncovering areas, wherein the optimizing is carried out 
temporal position of the next image in covering areas and at 
temporal position of the previous image in uncovering areas. 



14. (Amended) The apparatus as claimed in claim 13, wherfein the 
previous image is shifted over a fraction a times the candidate 
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vector, the next image Is shifted over 1 -a times the candidate 
vector and the fraction a may change within the image period 

15. (Amended) The apparatus as claimed in claim 13, wherein the 
criterion function is a match error which is minimized. 

16. (Amended) The apparatus as claimed in claim 14, wherein 
said apparatus further comprises a covering/uncovering deteclj:or for 
controlling the fraction a in the matching process. 



case 



17 . (Amended) The apparatus as claimed in claim 16, wher^ 
fraction a is set to 1 in case of covering and set to 0 in 
uncovering . 



wherein 



18. (Amended) The apparatus as claimed in claim 16, 
covering/uncovering detector decides on data in a previous linage 
the fraction a in the current estimation. 



19. (Amended) The apparatus as claimed in claim 13, wher^ 
velocity edge is determined, an occlusion area is marked 
said edge, and in said occlusion area, foreground velocity i 
replaced by background velocity or reversibly dependent on 
5 the occlusion area is a covering or uncovering area, the 



sign 



in the 
of 



the 
to 



in a 
i round 

3 

wjaether 
of 
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the foreground velocity and on which side of the velocity 
the foreground ie. 



edoe Xe 



where :in 



the 



10 



20. (Amended) The apparatus as claimed in claim 19. 
said apparatus further comprises calculation means for, at 
position 5, of a velocity edge, calculating 

• a first position 3c, in the previous (covering) or next 
(uncovering) image by shifting x, over the first vector at 
of . the edge 

• a second position in the previous (covering) or next 
(uncovering) image by shifting 5, over the second vector at 
other side of the edge 

• and a third intermediate position between and , 

• while finally, the vector fetched with Vg„ at the third 
in the previous (covering) or next (uncovering) image O) is 
in those regions of the image in the environment of the edge 
which no vector is projected, in case the background vector 



OTie side 



1:he 



p03 



15 should be filled in, and the vector chosen between D{x- 



£)(x+^^,n) which is most different from v,^ is filled in, in 
foreground vector v^^ should be filled in. 



it ion 
filled 
, to 

and 
case a 
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21* (Amended) The apparatus as claimed in claim 20, where 
intermediate position is {x^ + x^)/2. 



aa 
s to 



f or<;ground 



22. (Amended) The apparatus as claimed in claim 19, wherein 
said apparatus further comprises means for projecting two po4itions 
on either side of the edge to the previous (covering) or next 
(uncovering) image / in which a background velocity is identified 
5 a velocity which crosses the velocity discontinuity and project 
a foreground velocity in the previous picture, whereas a 
velocity projects to itself. 

23 » (Amended) The apparatus as claimed in claim 19, wherein 
said apparatus further comprises means £or testing near edgei^ 
whether the mentioned edge has moved over the first vector 
side of the edge, or over the second vector on the other sidi 
5 the edge, in case the edge moves with the first (second) vector, 
the second (first) vector is filled in those regions of the 
projected vector field in the environment of the edge, to which no 
vector is projected, in case a background vector v^^ should 
filled in, and the other vector is filled in, in case a foreground 
10 vector v^^ should be filled. 



oil 



wher i^in 



24. (Amended) The apparatus as claimed in claim 22, 
said apparatus further comprises verification means for veri 
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the crossing from a background region to a foreground region 
previous image by the match error of the vector in that block 



in the 



25. (Amended) An image display apparatus comprising apparjatus 
for detecting a motion vector as claimed in claim 13, means 
interpolating image parts connected to said detecting 
and a display device connected to the interpolating means 



for 



apparat us 
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(ix) Evidence Appendix 

There is no evidence which had been submitted under 37 C|.F.R, 
1.130, 1.131 or 1,132, or emy other evidence entered by the 
Examiner and relied upon by Appellants in this Appeal. 
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(x) Related Proceedings Appendix 

Since there were no Proceedings identified in section (i 
herein, there are no decisions rendered by a court or the B05 
any proceeding identified pursuant to paragraph (c) (!) (ii) of 
CF.R. 41.37- 
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